Software Implications for Multimodal User Interfaces
نویسندگان
چکیده
This paper discusses software considerations for multimodal user interfaces, that is, systems able to support human-to-human modalities of communication (such as gesture and natural language). A definition and a classification of multimodal systems is proposed and the distinction between multimodal and multimedia systems is clarified. Then multiagent models and techniques used in graphical user interfaces are compared with concepts and approaches developed in computer vision and natural language processing. This analysis leads the way to the definition of a general framework aimed at the integration of multiple modalities.
منابع مشابه
Complex Event Pattern Recognition Software for Multimodal User Interfaces
We describe how CERA, the Complex Event Recognition Architecture, was used to create a multimodal user interface for Skibbles, a memory game moderated by a mobile robot. We also announce the availability of open-source software that implements CERA, and how it can be used to build intelligent multimodal interfaces.
متن کاملTowards Multimodal User Interfaces Composition Based on UsiXML and MBD Principles
In software design, the reuse issue brings the increasing of web services, components and others techniques. These techniques allow reusing code associated to technical aspect (as software component). With the development of business components which can integrate technical aspect with HCI, the composition issue has appeared. Our previous work concerned the GUI composition based on an UIDL as U...
متن کاملEcological Interfaces: Extending the Pointing Paradigm by Visual Context
Following the ecological approach to visual perception, this paper presents an innovative framework for the design of multimodal systems. The proposal emphasises the role of the visual context on gestural communication. It is aimed at extending the concept of affordances to explain referring gesture variability. The validity of the approach is confirmed by results of a simulation experiment. A ...
متن کاملInteractive Systems & New Interface Technologies Engineering for Multimodal Human - Computer Interaction
1 Introduction Engineering for Human-Computer Interaction (EHCI) focuses on system development models, methods, processes and tools that enable teams to create effective user interfaces on-time and on-budget. Software engineering typically identifies four main classes of activity within a system development project: Planning, Analysis, Design, Evaluation. For the design activity, architectural ...
متن کاملInteraction Styles in Use of Automotive Interfaces
Natural language speech enabled interfaces offer a compelling choice of user interaction for the automotive market. Such interfaces demand only a basic understanding of the system model instead of memorizing keywords and predefined patterns. To utilize natural language interfaces optimally, designers need to better comprehend how people communicate their requests to express their intentions and...
متن کامل